What  I  really  want 


My  personal  information  to  be  accessible 

Published  information  should  find  me 
Usable  anywhere 

Others  can  use  what  I  have  learned 
(if  I  want  them  to) 
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Wide  Area  Information  Servers: 
A  Supercomputer  on  every  Desk 


Brewster  Kahle 
ing  Machines  Corporation 
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What  is  it? 

Electronic  Publishing 

(Or  publishing  over  wires) 


New  Communications  Technology  Problems 


BOOKS 

Talonronhs 

1 6i6(jrcipn> 

Telephone 

c.it?i#iruiiiu 

Publishing 

Experts  only 

Monks 

www  \J§  mm\%9 

Operators 

Professional 
searchers 

Distribution  is  hard 
and  expensive 

Vellum  is 
calfskin 

Telephones  on 
barb  wire 

$1/minute  over 
obscure  modems 

Different  interfaces 

1000's  of  languages 

ill  c Lil \JfJts  aiuilc 

Switching  was 

manual 

IIIOIIUOI 

//query  (W5) 
inform? 

f f  fll/f f f f  a 

Material  is 
intractable 

Scrolls  and  manu- 

9cnpi9  were  ctufjui 

as  random  access 
as  musical  scores 

No  white  pages 

600  databases 

on  Dialon 

~7  Terabyte 
140Gbyte  at  DJ 
80GB  card  catalog 
at  RLG 

Business  model 
needed 

Centralized  printing 

Pay  per 
minute 

Not  understood 

■MR 
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Navigation  Techniques:  Paper 

•  Alphabetical  Listings  (dictionary,  Encyclopedia) 

•  Indices  (back  of  the  book  and  Readers  Guide) 

•  Table  of  Contents  (outlining) 

•  Citation  index 

•  "Tree  of  Knowledge" 

•  Have  you  read  any  good  books  lately? 
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Navigation  Techniques:  Computers 

•  Hierarchical  File  Systems 

•  Unix  "find"  and  "grep",  Mac  "find  file" 

•  Gopher,  Magellan,  ON  Location 

•  Boolean  query  systems  (...within  5  words  of...) 

•  Static  Hypertext  links  (see  also  pointers) 
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Navigation  Techniques:  WAIS 


English  language  questions  and  relevance 

lcM£UUclul\ 

-  Question-answer  dialog 

-  Similar  to  Newspapers:  "More  on  page  5" 

-  Dynamic  Hypertext  Links 

2  level  search: 

-  Directory  of  servers  (server  like  any  other) 

-  Servers  themselves 
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Wide  Area  Information  Server 

Architecture 


Dow  Jones 


)irectory  of 

Servers 


Gateways 
>  other  nets 


TV  Guide 
etc. 


Z39.50  over 

X.25,  TCP/IP,  Modem 
Open  Interconnection 
Public  Protocol 


LAN  Server 


Image  I  Private 
Servers  I  Servers 


Users  Needs: 
Selecting  Servers 
Answering  Questions 
Organizing  Responses 


Architecture  Issues: 
Scalability 
Security 

Business  model  for  servers 
Reliable  Access 
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Demonstration  System  Structure 


WAIS 


I 


LAN 


Server 


x.25 
Z39.50 
9600Baud 


Connection 
Machine 


III 

— *  ™ 


Workstations 


•  •  • 


HI 


Operations: 

Archiving 

Queries 

Retrieval 
IR  Type: 

Broadcast 

Query  by  Example 
Databases: 

Wall  St  Journal 

Barron's 

400  Business  Mags 


CM: 

Operations: 

Queries 
IR  Type: 

enhanced  relevance 

feedback 
Databases: 

DowVision  and  memo's, 

mail,  word  processor  files 


Mac: 

Operations: 

Human  Int 

Retrieval 

Queries 

"Caching"  Docs 

User  Profiles 
IR  Type: 

Query  by  example 
Databases:  . 

Personal  Text 

Cached  data 
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WAIS  Hardware  Components 
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WAIS  Clients 

•  Busy  24  hours  a  day  finding  information 

•  Ponder  all  indications  of  the  preferences  of  its  user 

•  Gossip  with  other  clients  about  their  discoveries 

•  Scours  the  world  (within  a  budget)  to  find 
new  sources 

•  Current  implementations  on  PC,  Macintosh, 
X  Windows,  NeXT,  dumb  terminal  (dial-up) 
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WAIS  Protocol 

•  Based  on  NISO  Z39.50  international  standard 

•  Flexible  —  separates  clients  from  servers 

•  Search:  (words,  docjds,  databases) 
returns  list  of:  (headline,  score,  docjd,  types) 

•  Retrieval:  (docjd,  type,  start,  end) 
returns:  data  of  specified  type 

•  Docjd:  An  ISBN  for  the  Electronic  Age 

•  Server  Description  Structure  for  the  Directory 
of  Servers 


mmmmmmssiimmmi 
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How  Standard  Protocol 
can  Provides  Security 


Users  do  not  login  to  server,  but  search  only 

through  application  layer  protocol  (Z39.50). 

Server  controls  access  to  data. 

Network  layers  below  application,  or  application 
layer  handles  authentication,  encryption,  billing. 
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The  WAIS  Protocol  is  WAIS 


Client 


Supports  any  search  syntax 

Supports  sophisticated  clients  — 
puts  intelligence  in  the  user's  hands 

Clients  can  run  on  any  platform 

Multiple  servers  in  a  single  search 

Retrieve  any  kind  of  data:  text,  graphics,  video,... 
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Connection  Machine  Server 

1-100G Bytes  (and  getting  bigger) 
Supports  thousands  of  users 
Automatic  Indexing 

Uses  words  and  phrases  in  question  to  find 
appropriate  documents  with  relevance  feedback, 
weighted  term 

Supports  Boolean  Queries 

Cost  effective  hardware  alternative  to  mainframes 
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Data  Parallelism: 

Searching  all  the  Documents  at  Once 


Stadium 
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Boolean  Search 


Retrieve  documents  containing 
specific  combinations  of  words 

■mam 


Conceptual  Search 


Explore  a  set  of  documents 
containing  related  concepts 

Thinking  Machines  Corporation 


Boolean  Query 


(Japanese  OR  Japan)  AND 

(building  OR  buildings  OR  (Real  AND  Estate)  AND 
(Manhattan  OR  (New  AND  York) 


Have  you  been  paying  attention?... 
Freer  Finance:  U.S.  Regulators  Move... 
REAL  ESTATE:  California  Initiatives... 
First  Boston  Said  To  Agree  on  Sale  Of... 
Exxon,  Rockefeller  Group  to  Sell  Site... 
What's  News-Business  and  Finance 


Hard  to  Use: 

Complex  Syntax 


Poor  Results: 

The  wrong  information 

No  ranking  of  results 
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Conceptual  Search:  Phase  1 


1.  Time  Acts  to  Cut  Magazine  t  nsts... 

2.  First  Boston  Said  To  Agree  on  Sale, , . 

3.  Have  You  Been  Paying  Attention? 

4.  Exxon,  Rockefeller  Group  to  Sell  Site. . , 

5.  Hard  Sell:  Real  Estate  Developers... 

6.  What's  News-Business  and  Finance. . . 

7.  Integrated  Resources  Buys  Loft  Building. . . 


Easy  to  Use:  I 
No  Syntax 

Options: 

What  do  you  want 
to  follow  up? 
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Conceptual  Search:  Phase  2 


First  Boston  Said  To  Agree  on  Sale... 

Exxon,  Rockefeller  Group  to  Sell  Site... 


1.  Bids  for  Exxon  Building  in  New  York. . . 

2.  Time  Acts  to  Cut  Magazine  Costs. . . 

3.  Hard  Sell:  Real  Estate  Developers. . . 

4.  Time  Inc.  Sells  Its  45%  Interest... 

5.  Citicorp  Unit  Moves  to  Foreclose  on. . . 

6.  Litigious  Landlords:  Legal  Maneuvers 


Relevance  Feedback: 
I  like  these;  show 
sue  more. 


Improved  results: 

Articles  on  related 
topics  are  found. 

Results  are  ranked. 


MHBi 
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Query  Broadcast  To  Database 
on  Connection  Machine  System 


Document  Units  Scores 


Tripoli 


Libyan 


PLO 


bomb 
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i0[    in|     *t|    eo|     csj|    t^|  01 


Document  Retrieval  Performance 


Current  algorithm  limits: 

~2GBwith  512MBCM-2 
~8  GB  with  2  GB  CM-2 
-25  GB  with  8  GB  CM-2 


High  recall        1  Stanfill  and  Kahle, 

L  see  Communications  of  the  ACM , 

u.  .         .  .       J  December  1986 

High  precision  J 


•  «  1  sec.  response 

•  Much  larger  DBs  searchable  with  CM-5 

and  inverted  index  algorithms:  100s  to  1000s  of 
Gigabytes 
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jWAIStation! 


WAIStation: 
Active  Database  Sources, 
Saved  Questions 


Questions 

<•>  CM  applications 

O 

? 

B 

CM  Apps  Question 

Encyclopedia 

? 

Library  question 

<•>  King  James  Bible 

? 

I 

Encyclopedia  Q 

<•>  Macintosh  Hard  Disk 

? 

Patent  Q 

<•>  TMC  Business  email 

■ 

TMC  Bus.  Email  0 

<•>  TMC  Library  Catalog 

? 

TMC  Fun  Q 

<•>  VjffSt.Jborruf 

? 

■ 

Montvale  0 

<•>  World  Factbook 

7 

World  Factbook  0 

? 

■ 

poetry  q 

O 

■ 

Bible  Q 
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tfmmmmmmmm^mmi  ;  mmmmmms^s^si.  ^^i*,.,,^ 

Select  Data  Source 


Sources 


<•>  CM  applications 
<•>  Encyclopedia 
<•>  King  James  Bible 
<•>  Macintosh  Hard  Disk 
<•>  TMC  Business  email 
<•>  TMCLibran.  r-*-- 

<•>  World  Factbook 


Question-1 


Look  for  documents  about 


Which  are  sim^r  to  In  these  sources 


M 


<•>  Miff  St.  Jburnif 


Results 


o 


Run 


0 


■■■Ilfi5-?1^ 
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Run  Initial  Query 


Question-1 


Look  for  documents  about 


recent  developments  in  personal 
computers!  


Vhich  are  similar  to  In  these  sources 


<&>  faff  St.  Owrw/ 


<> 


Run 


O 


Results 


***  Compaq  Computer  Directors  Approve  2-for-1  Stock  Split  ^ 
***  International:  Bull  Agrees  to  Pay  Zenith  $15  Million  to  Enf 
***  AT&T  Set  to  Announce  Memorex  Computer  Accord 
***  Technology  Brief  —  International  Business  Machines :  Prk 
Business  Brief  —  Data  General  Corp. :  Four  Models  Are  Ur 


Technology  :  Computer  Firm.-  See  the  'w'ritinq  on  the  Scree 


***  Retailing:  Businessland  Enters  Japan,  Aided  by  4  Big  Loca 
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Click  a  Headline  to  Display  a  Document 


Ui 


Question- 1 


Look  for  documents  about 


recent  developments  in  personal 
computers!  


g  ED 


Vhich  are  similar  to  In  these  sources 


<•>  KiHSt.  Jkmrrul 


Results 


Compaq  Computer  Directors  Approve  2-for~1  Stock  Split 
International :  Bull  Agrees  to  Pay  Zenith  $15  Million  to  Em 
AT&T  Set  to  Announce  Memorex  Computer  Accord 
Technology  Brief  —  international  Business  Machines :  Pri<  iiiiil 
Business  Brief  —  Data  General  Corp, :  Four  Models  Are  Uf 


ft 


[□I  Technology:  Computer  Firms  See  the  Writing  i  iHl 


International  Business  Machines  Corp.,  Apple  Computer  Inc. 
and  other  big  computer  makers  are  staking  out  positions  i  n 
the  nascent  market  for  "note- pad  computers,''  small  machines 
that  let  users  enter  data  by  writing  rather  than  tapping 
keys.  The  note  pads  typically  recognize  numbers  and  letters 
printed  on  a  screen  with  a  special  pen  and  convert  them  into 
conventional  electronic  characters.  The  information  is  then 
stored  for  later  transfer  to  a  personal  computer  or  a 
company's  main  computers. 

The  size  of  the  market  for  note- pad  computers  isn't  clear, 
but  Infocorp,  a  Santa  Clara,  Calif.,  market- research  firm, 
estimates  the  market  will  grow  to  3.4  million  units  sold  in 
1 995  from  22,000  units  this  year.  Only  one  company,  Tandy 
Corp 's  Grid  Systems  unit,  currently  sells  note- pad  computers 
in  the  U.S.;  its  model,  introduced  last  September,  is  priced 
at  $3,000.  But  new  ventures  are  expected  to  introduce  several 
note-pad  machines  this  year.  And  already,  big  computer  makers 
are  fighting  quietly  for  control  over  software  standards  for 
these  gadgets,  which  require  different  programs  from  those 
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Relevance  feedback: 
"Find  me  more  like  this  one 


55 


Question-1 


Look  for  documents  about 


recent  developments  in  personal 
computers 


Run 


Vhich  are  similar  to  In  these  sources 


ID  Technology  :  Coi  ^ 


<•>  VjrrSt.Jburruf 


o 


Compaq  Computer  Directors  Approve  2-for-1  Stock  Split  ^ 
***  International :  Bull  Agrees  to  Pay  Zenith  $1 5  Million  to  En< 

AT&T  Set  to  Announce  Memorex  Computer  Accord 
***  Technology  Brief  —  International  Business  Machines :  Pri< 
**»  Business  Brief —  Data  General  Corp.:  Four  Models  Are  Un  m 


T**hri*k»jg  :  Computer  Firm.-  S**  th*  '••vritinq  on  th* 


[?|  ***  Retailing:  Businessland  Enters  Japan,  Aided  by  4  Big  Loca 
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Relevance  Feedback  of  Paragraph 


Technology:  Computer  Firms  See  the  Writing  > 


Computer  makers  are  scrambling  to  cash  In  on  people  who 
find  the  pen  mightier  than  the  keyboard. 

International  Business  Machines  Corp.,  Apple  Computer  Inc. 
md  other  big  computer  makers  are  staki  ng  out  positions  i  n 
ie  nascent  market  for  "note- pad  computers,"  small  machines 
*t  let  users  enter  data  by  writing  rather  than  tapping 
.  is.  The  note  pads  typically  recognize  numbers  and  letters 
pted  on  a  screen  with  a  special  pen  and  convert  them  into 


Question-1 


cor 


Look  for  documents  about 


Th|||Jjecent  developments  in  personal 
Computers 


esti  iti 
1995 


Which  are  similar  to  In  these  sources 


=  Technology  :  Coi  O"    ^  Waff  St.  Uwnu/ 


Results 


ID 
i 
i 

m 
i 
m 
a 

J2L 


Pri< 


Compaq  Computer  Directors  Approve  2-for-1  Stock  Split 
International:  Bull  Agrees  to  Pay  Zenith  $15  Million  to  En< 
AT&T  Set  to  Announce  Memorex  Computer  Accord 
Technology  Brief  —  International  Business  Machines 
Business  Brief —  Data  General  Corp.:  Four  Models  Are  Ur 
Technology  :  Computer  Firms  See  the  Writing  on  the  Scree 
Retailing:  Businessland  Enters  Japan,  Aided  by  4  Big  Loca 
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Chaining"  of  Questions  to  Follow  a  Tangent 


Question-1 


Look  for 


about 


recent  developments  in  personal 
computersl  


Which  are  similar  to  In  these  sources 


=  Technology  :  Coi  ^ 


<•>  YjffSt.Jbamif 


Results 


i 
i 
B 
B 


En<<> 


International:  Bull  Agrees  to  Pay  Zenith  $15  Million  to 
AT&T  Set  to  Announce  Memorex  Computer  Accord 
Technology  Brief  —  International  Business  Machines :  Pri( 
Business  Brief  —  Data  General  Corp. :  Four  Models  Are  Un 
Technology  :  Computer  Firms  See  the  Writing  on  the  Scree 


F' •"■tjiliivj  Eurin^'jlsrid  Ent^rj  . 


nldrd  bu  4  Eld  1 


Coi 


Question-2 


Look  for  documents  about 


Vhich  are  similar  to  In  these  sources 


HI  Retailing :  Busin  ^ 


<S>  VjffSt.  U>urruf 


Results 


B 

H 

a 

El 

b 

o 

El 


Retailing:  Businessland  Enters  Japan ,  Aided  by  4  Big  Loca 
What's  News  —  Business  and  Finance 
Technology  :  Computer  Makers  Agree  on  a  Standard  For  Nfer 
Inside  Track :  Businessland  Directors  Take  a  Loss  And  Tra 
Technology  &  Health :  Businessland  To  Report  Loss  For  3r 
Technology  :  U.S.  Computer  Maker  Takes  on  NEC  on  Its  Ow 
Technology  :  Computer  Firms  See  the  Writing  on  the  Scree 
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TMC  Internet  Release 

CM  product  for  TCP/IP  (complete  server) 

Example  User  interfaces  for  free  (no  support) 
Macintosh,  Gnu  Emacs,  Xwsndows 

Example  unfx  server  software  to  create  servers 

Directory  of  Servers  on  the  internet  at  least 
through  s92 

160  Servers  now:  Weather  Maps,  patents,  journal 
abstracts,  email  archives,  usenet  recipies,... 

Free  Software  via  FTP  from  Think.com:/wais/* 

Mailing  list:  wais-discussion-request@think.com 
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WAIS  Uses 


Over  1 0,000  users  on  the  Internet 

Users  in  24  Countries:  Mexico,  Singapore, 

Finland,  Australia,  etc 

160  Databases  served  from  9  Countries: 
Norway,  Canada,  UK,  etc.  Average  3  new 
databases  registered  per  week. 
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WAISUses: 
Campus  Wide  Info  Servers 

•  Class  catalog  and  schedule 

•  Campus  events:  movies,  sports 

•  Job  listings 

•  Library  catalog 

•  Phone  book 

•  Professor  research  interests 

•  Past  theses 


[  sol.acs.unt.edu] 
[  xantos • uio . no ] 

[      next 2 . oit • unc • edu ] 


UNTComputerDoc 
UiO_Publications 
ibm .  pc .  FAQ 
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WAIS  Uses:  Libraries 


•  Easy  to  use  card  catalog 

•  Remote  use  from  home  or  office 

•  Pictures,  full  text,  scanned  documents 


[pegun. law. Columbia. e]  columbia-law-library-catalog 
[pegun . law. Columbia • e ]  columbia-spanish-law-catalog 
[  quake.think.com]     tmc- library 
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WAIS  Uses:  Biology 


•  Journal  Abstracts 

•  Sequence  archives 

•  Images 

Currently  over  20  Biology  databases  in 
Finland,  Netherlands,  and  US 


cmns . think . com] 
bio .vu.nl] 
genbank . bio . net ] 
wais.funet.fi] 
wais . funet.fi] 
wais . funet.fi] 


Molecular-biology 
biology-compounds 
biology- j  ournal -contents 
bionic-ai-researchers 
bionic-directory-of -servers 
bionic-enzyme 
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WAIS  Uses:  Chemistry 
CORE  Project 

•  All  published  chemistry  (8  years  all  ACS) 

•  Scanned  pictures,  ascii  text 

•  Optical  jukebox  mass  storage 

•  Connection  Machine  /  Newton  search  engines 

Project  of  Bellcore,  ACS,  Chem  Abstracts, 
OCLC,  Cornell,  and  Thinking  Machines 

cu j  o . curt in . edu . au ]  chem-eng-current-contents 
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WAIS  Uses: 
Business  Executives 


•  Dow  Jones  information 

•  Corporate  information 

•  Personal  information 

Project: 

KPMG,  Apple,  Thinking  Machines,  Dow  Jones 

[  cmns.think.com]     wall -street- journal -sample 

[  think.com]     Business -email 
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WAIS  Uses: 
Medical  Researchers/Doctors 


Medical  papers 

Storing  and  matching  patient  records 
Remote  connections  to  specialized  databases 

[  wais.funet.fi]  bionic-databases-limb 
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WAIS  Uses: 
Community  Information 

•  Dial-up  users:  no  network  required 

•  Directories  of  services  or  facilities 

•  Education  and  entertainment 


[  quake.think.com] 

[  sol . acs • unt . edu ] 

[  quake . think . com] 

[  lambada.oit.unc.edu] 


int e  rne t - r e s our ce -gu ide 

online-libraries 

weather 

nsf -bulletins 


mmm 
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Conclusion 


Electronic  Publishing  can  fill  niches  now. 

Companies  are  positioning  themselves  now 
(workstations,  server,  and  info  providers). 

Thinking  Machines  is  the  "Engine  of  the 

Information  Industry." 
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